Deblocking filters are used in video coding standards in order to combat blocking artifacts. The blocking artifacts arise because the original video is split into blocks which are processed relatively independently. The blocking artifacts can arise due to different intra prediction of blocks, quantization effects and motion compensation. Two particular variants of deblocking are described below.
These descriptions assume a vertical block boundary or border. The process is also done in the same way for horizontal block boundaries.
H.264 Deblocking
In state of the art video coding such as H.264 there is an adaptive de-blocking filter/loop filter after prediction and residual reconstruction, but before storage of the reconstruction for later reference when encoding or decoding subsequent frames. The deblocking filtering consists of several steps such as filter decisions, filtering operations, a clipping function and changes of pixel values. The decision to filter the border or not is made based on evaluating several conditions. Filter decisions depend on macro block (MB) type, motion vector (MV) difference between neighboring blocks, whether neighboring blocks have coded residuals and on the local structure of the current and/or neighboring blocks.
The amount of filtering for a pixel depends on the position of that pixel relative to the block boundary and on the quantization parameter (QP) value used for residual coding. Here below a to h represent pixel values across a vertical block boundary.                abcd|efgh        
The filter decision is based on comparing three pixel differences with three thresholds. The thresholds are adapted to the QP.
If the following conditions are fulfilled the filtering is done:abs(d−e)<thr1,abs(c−d)<thr2, andabs(e−f)<thr2where thr1 and thr2 are functions of QP.
There are two filtering modes in H.264. In the first filtering mode (normal filtering), filtering can be described with a delta value that the filtering changes the current pixel value with. The filtering for the pixel closest to the block boundary is:d′=d+delta and e′=e−deltawhere delta has been clipped off to a threshold ±thr3 to a value that is constrained by the QP. d′ is here the pixel value at position d after filtering and e′ is the pixel value after filtering at position e. More filtering is allowed for high QP than for low QP.
Clipping can be described as:delta_clipped=max(−thr3,min(thr3,delta))where thr3 is controlling the filter strength. A larger value of thr3 means that the filtering is stronger, which in turns means that a stronger low-pass filtering effect will happen.
The filter strength can be increased if any of the following two conditions also holds:abs(b-d)<thr2 and abs(e-g)<thr2
The filter strength is adapted by clipping the delta less, e.g. to allow for more variation.
The second filtering mode (strong filtering) is applied for intra macroblock boundaries only, when the following condition is fulfilled:abs(d−e)<thr¼.
The thresholds thr1, thr2 and thr3 are derived from table lookup using QP as index. Each slice can contain modifications of thr2 and thr3 using slice_beta_offset_div2 and thr1 using slice_alpha_c0_offset_div2. The slice parameters 2×slice_beta_offset_div2 and 2×slice_alpha_c0_offset_div2 are added to the current QP index before table lookup of thr2/thr3 and thr1 respectively.
Deblocking in HEVC Draft
Here below p0 to p3 and q0 to q3 represent pixel values across a vertical block boundary.                p3p2p1p0|q0q1q2q3         
In the draft HEVC specification, the deblocking filter works differently than H.264. The filtering is performed if at least one of the blocks on the side of the border is intra, or has non-zero coefficients, or the difference between the motion vector components of the blocks is greater than or equal to one integer pixel. For example, if one is filtering the border between the blocks A and B below, then the following condition should satisfy for the block boundary to be filtered:
                                                                                                              p                    ⁢                                                                                  ⁢                                          3                      0                                                                                                            p                    ⁢                                                                                  ⁢                                          2                      0                                                                                                            p                    ⁢                                                                                  ⁢                                          1                      0                                                                                                            p                    ⁢                                                                                  ⁢                                          0                      0                                                                                                            q                    ⁢                                                                                  ⁢                                          0                      0                                                                                                            q                    ⁢                                                                                  ⁢                                          1                      0                                                                                                            q                    ⁢                                                                                  ⁢                                          2                      0                                                                                                            q                    ⁢                                                                                  ⁢                                          3                      0                                                                                                                                        p                    ⁢                                                                                  ⁢                                          3                      1                                                                                                            p                    ⁢                                                                                  ⁢                                          2                      1                                                                                                            p                    ⁢                                                                                  ⁢                                          1                      1                                                                                                            p                    ⁢                                                                                  ⁢                                          0                      1                                                                                                            q                    ⁢                                                                                  ⁢                                          0                      1                                                                                                            q                    ⁢                                                                                  ⁢                                          1                      1                                                                                                            q                    ⁢                                                                                  ⁢                                          2                      1                                                                                                            q                    ⁢                                                                                  ⁢                                          3                      1                                                                                                                                        p                    ⁢                                                                                  ⁢                                          3                      2                                                                                                            p                    ⁢                                                                                  ⁢                                          2                      2                                                                                                            p                    ⁢                                                                                  ⁢                                          1                      2                                                                                                            p                    ⁢                                                                                  ⁢                                          0                      2                                                                                                            q                    ⁢                                                                                  ⁢                                          0                      2                                                                                                            q                    ⁢                                                                                  ⁢                                          1                      2                                                                                                            q                    ⁢                                                                                  ⁢                                          2                      2                                                                                                            q                    ⁢                                                                                  ⁢                                          3                      2                                                                                                                                        p                    ⁢                                                                                  ⁢                                          3                      3                                                                                                            p                    ⁢                                                                                  ⁢                                          2                      3                                                                                                            p                    ⁢                                                                                  ⁢                                          1                      3                                                                                                            p                    ⁢                                                                                  ⁢                                          0                      3                                                                                                            q                    ⁢                                                                                  ⁢                                          0                      3                                                                                                            q                    ⁢                                                                                  ⁢                                          1                      3                                                                                                            q                    ⁢                                                                                  ⁢                                          2                      3                                                                                                            q                    ⁢                                                                                  ⁢                                          3                      3                                                                                                          A              ⁢                                                          ⁢              B                                ⁢                                          ⁢                                    dp              ⁢                                                          ⁢              0                        =                                                                          p                  ⁢                                                                          ⁢                                      2                    0                                                  -                                  2                  ×                  p                  ⁢                                                                          ⁢                                      1                    0                                                  +                                  p                  ⁢                                                                          ⁢                                      0                    0                                                                                              ⁢                                          ⁢                      dp            ⁢                                                  ⁢            3                    =                                                                p                ⁢                                                                  ⁢                                  2                  3                                            -                              2                ×                p                ⁢                                                                  ⁢                                  1                  3                                            +                              p                ⁢                                                                  ⁢                                  0                  3                                                                                ⁢                                  ⁢                              dq            ⁢                                                  ⁢            0                    =                                                                q                ⁢                                                                  ⁢                                  2                  0                                            -                              2                ×                q                ⁢                                                                  ⁢                                  1                  0                                            +                              q                ⁢                                                                  ⁢                                  0                  0                                                                                ⁢                                  ⁢                              dq            ⁢                                                  ⁢            3                    =                                                                q                ⁢                                                                  ⁢                                  2                  3                                            -                              2                ×                q                ⁢                                                                  ⁢                                  1                  3                                            +                              q                ⁢                                                                  ⁢                                  0                  3                                                                                ⁢                                  ⁢                              dpq            ⁢                                                  ⁢            0                    =                                    dp              ⁢                                                          ⁢              0                        +                          dq              ⁢                                                          ⁢              0                                      ⁢                                  ⁢                              dpq            ⁢                                                  ⁢            3                    =                                    dp              ⁢                                                          ⁢              3                        +                          dq              ⁢                                                          ⁢              3                                      ⁢                                  ⁢                  dp          =                                    dp              ⁢                                                          ⁢              0                        +                          dp              ⁢                                                          ⁢              3                                      ⁢                                  ⁢                  dq          =                                    dq              ⁢                                                          ⁢              0                        +                          dq              ⁢                                                          ⁢              3                                                          (        1        )            
The variable d is derived as follows:d=dpq0+dpq3  (2)
The deblocking filtering is performed on the block boundary for lines i=0 . . . 3 if the following condition holds:d<β  (3)where β depends on QP. In the draft HEVC specification, there is a table, see Table 1 below, for looking up the value of β using QP as the table index. β increases with increasing QP.
TABLE 1Derivation of threshold variables β and tc from input QPQP0123456789101112131415161718β0000000000000000678tc0000000000000000001QP19202122232425262728293031323334353637β9101112131415161718202224262830323436tc1111111122223333444QP383940414243444546474849505152535455β384042444648505254565860626464646464tc5566789910101111121213131414
If the condition in equation (3) above is fulfilled and filtering is done between blocks A and B, one of two types of filtering (weak or strong filtering) is performed. The choice between the strong and the weak filtering is done separately for each line depending on the following conditions. For lines i=0,3, strong filtering is performed if all the following conditions are true, otherwise, weak filtering is performed:2×dpqi<(β>>2)  (4)and (|p3i−p0i|+|q0i−q3i|)<(β>>3)  (5)and |p0i−q0i<((5*tc+1)>>1),  (6)where tc and β depend on QP as shown in Table 1. The tc is calculated as tc×(QP+2) when A or B has PredMode==MODE_INTRA.
The two filtering modes (weak and strong filtering) in the HEVC draft look like in the following:
Weak Filtering
Weak filtering is performed based on the above conditions. The actual filtering works by computing an offset (Δ) for each of the lines i that the weak filter has been selected for. The following weak filtering procedure is applied for every line, where it has been chosen. In the following algorithm, the variables p0 . . . p2 and q0 . . . q2 are assigned the following values (from row/column i) where p0=p0i, p1=p1i, p2=p2i, q0=q0i, q1=q1i, q2=q2i. One can see that if the weak filtering is performed, one to two pixels are modified at each side of the block boundary:
                                              ⁢                              Δ            =                                          (                                                      9                    ×                                          (                                                                        q                          ⁢                                                                                                          ⁢                          0                                                -                                                  p                          ⁢                                                                                                          ⁢                          0                                                                    )                                                        -                                      3                    ×                                          (                                                                        q                          ⁢                                                                                                          ⁢                          1                                                -                                                  p                          ⁢                                                                                                          ⁢                          1                                                                    )                                                        +                  8                                )                            ⪢              4                                ⁢                                          ⁢                                          ⁢                      if            ⁢                                                  ⁢                          (                                                abs                  ⁡                                      (                    Δ                    )                                                  <                                  10                  ×                                      t                    c                                                              )                                ⁢                                          ⁢                                          ⁢                      {                                                  ⁢                                                  ⁢                          Δ              =                                                Clip                  ⁢                                                                          ⁢                  3                  ⁢                                      (                                                                  -                                                  t                          c                                                                    ,                                              t                        c                                            ,                      Δ                                        )                                    ⁢                                                                          ⁢                                                                          ⁢                  p                  ⁢                                                                          ⁢                                      0                    ′                                                  =                                                      Clip                    ⁢                                                                                  ⁢                                          1                      Y                                        ⁢                                          (                                                                        p                          ⁢                                                                                                          ⁢                          0                                                +                        Δ                                            )                                        ⁢                                                                                  ⁢                                                                                  ⁢                    q                    ⁢                                                                                  ⁢                                          0                      ′                                                        =                                      Clip                    ⁢                                                                                  ⁢                                          1                      Y                                        ⁢                                          (                                                                        q                          ⁢                                                                                                          ⁢                          0                                                -                        Δ                                            )                                        ⁢                                                                                  ⁢                                                                                  ⁢                                          if                      ⁡                                              (                                                  dp                          <                                                      (                                                          β                              +                                                              (                                                                  β                                  ⪢                                  1                                                                )                                                                                      )                                                    ⪢                          3                                                )                                                              ⁢                                                                                  ⁢                                                                                  ⁢                                          {                                                                                          ⁢                                                                        Δ                          ⁢                                                                                                          ⁢                          p                                                =                                                                              Clip                            ⁢                                                                                                                  ⁢                            3                            ⁢                                                          (                                                                                                -                                                                      (                                                                          tc                                      ⪢                                      1                                                                        )                                                                                                  ,                                                                  tc                                  ⪢                                  1                                                                ,                                                                                                      (                                                                                                                  (                                                                                                                              (                                                                                                                                          p                                                ⁢                                                                                                                                                                                                  ⁢                                                2                                                                                            +                                                                                              p                                                ⁢                                                                                                                                                                                                  ⁢                                                0                                                                                            +                                              1                                                                                        )                                                                                    ⪢                                          1                                                                                )                                                                            -                                                                              p                                        ⁢                                                                                                                                                                  ⁢                                        1                                                                            +                                      Δ                                                                        )                                                                    ⪢                                  1                                                                                            )                                                        ⁢                                                                                                                  ⁢                                                                                                                  ⁢                            p                            ⁢                                                                                                                  ⁢                                                          1                              ′                                                                                =                                                      Clip                            ⁢                                                                                                                  ⁢                                                          1                              Y                                                        ⁢                                                          (                                                                                                p                                  ⁢                                                                                                                                          ⁢                                  1                                                                +                                                                  Δ                                  ⁢                                                                                                                                          ⁢                                  p                                                                                            )                                                                                                                          ⁢                                                                                          ⁢                                                                                          }                                                                                                                              (        7        )                                                          ⁢                              if            ⁡                          (                              dq                <                                  (                                      β                    +                                          (                                              β                        ⪢                        1                                            )                                                        )                                ⪢                3                            )                                ⁢                                          ⁢                                          ⁢                      {                                                  ⁢                                          Δ                ⁢                                                                  ⁢                q                            =                                                Clip                  ⁢                                                                          ⁢                  3                  ⁢                                      (                                                                  -                                                  (                                                      tc                            ⪢                            1                                                    )                                                                    ,                                              tc                        ⪢                        1                                            ,                                                                        (                                                                                    (                                                                                                (                                                                                                            q                                      ⁢                                                                                                                                                          ⁢                                      2                                                                        +                                                                          q                                      ⁢                                                                                                                                                          ⁢                                      0                                                                        +                                    1                                                                    )                                                                ⪢                                1                                                            )                                                        -                                                          q                              ⁢                                                                                                                          ⁢                              1                                                        -                            Δ                                                    )                                                ⪢                        1                                                              )                                    ⁢                                                                          ⁢                                                                          ⁢                  q                  ⁢                                                                          ⁢                                      1                    ′                                                  =                                  Clip                  ⁢                                                                          ⁢                                      1                    Y                                    ⁢                                      (                                                                  q                        ⁢                                                                                                  ⁢                        1                                            +                                              Δ                        ⁢                                                                                                  ⁢                        q                                                              )                                                                        ⁢                                                  ⁢                                                  }                          ⁢                                  ⁢                                  }                            (        8        )            where Clip is defined as x′=Clip3(A, B, x): x′=x or if x<A x′=A or if x>B x′=B and Clip1Y(x)=Clip3(0, (1<<BitDepthY)−1, x), where BitDepthY is the bit depth, for example 8 or 10.
Strong Filtering
Strong filtering mode is performed for a line i of pixels by the following set of operations, where p0=p0i, p1=p1i, p2=p2i, q0=q0i, q1=q1i, q2=q2i:p0′=Clip3(p0−2×tc, p0+2×tc,(p2+2×p1+2×p0+2×q0+q1+4)>>3)p1′=Clip3(p1−2×tc, p1+2×tc,(p2+p1+p0+q0+2)>>2)p2′=Clip3(p2−2×tc, p2+2×tc,(2×p3+3×p2+p1+p0+q0+4)>>3)q0′=Clip3(q0−2×tc, q0+2×tc,(p1+2×p0+2×q0+2×q1+q2+4)>>3)q1′=Clip3(q1−2×tc, q1+2×tc,(p0+q0+q1+q2+2)>>2)q2′=Clip3(q2−2×tc, q2+2×tc, (p0+q0+q1+3×q2+2×q3+4)>>3)
The decision for applying a strong deblocking filter in HEVC based on equations (4)-(6) makes sure that the strong deblocking filtering is only applied when the signal on both sides of the block boundary is flat. However, the prior art filter decision of whether to apply strong or weak deblocking filtering has disadvantages in terms of selecting weak deblocking filtering for some blocks and block boundaries even when strong deblocking filtering might be more appropriate. Hence, the prior art filter decision can result in insufficient deblocking filtering, which causes a reduction in subjective and/or objective quality of the picture and video.